This application claims benefit of Japanese Patent Applications Nos. 2001-165124 filed on May 31, 2001 and 2002-138044 filed on May 14, 2002, the contents of which are incorporated by reference thereto.
The present invention relates to defective image compensation systems and methods, which are assembled in or applied in combination to many solid-state imaging systems applied to digital cameras and television cameras.
In solid-state imaging elements constituted by CCDs or like semiconductor elements, it is difficult to uniformly form semiconductor crystal over a whole image pick-up area thereof. This gives rise to local formation of crystal defects, and these defective portions lead to a phenomenon of thermal charge generation, that is, these portions become defective pixels. Furthermore, in constructing a solid-state element in conformity to, for instance, a ⅔ inch image format, one pixel has a size of substantially 5 μm on each square side. Such minute pixels become defective pixels merely due to slight dust attachment to their surface.
In addition, the output of a defective pixel of the type, in which a fixed bias is superimposed on an electric signal corresponding to the incident light intensity, appears as a white point on the monitor screen. On the other hand, the output of a defective pixel of low light sensitivity appears as a black point on the monitor screen.
The compensation for defective pixels is usually executed on the basis of data of four pixels located around each defective pixel. More specifically, the average value of signals of four pixels adjacent to the defective pixel in horizontal and vertical line directions is substituted for the image output data of the defective pixel position. This compensation is based on an assumption that the data of the adjacent pixels have high correlation to the defective pixel position. Actually, such a process hardly leads to resolution deterioration.
In the case of the color imaging element, the term “adjacent to” has a meaning pertinent to pixels, which are pixels of the same color as the color of the defective pixel (i.e., pertinent color to the position of the defective pixel) and adjacent to the defective pixel in the horizontal or vertical direction. Also, although pixels which are physically closest to the defective pixel are usually of different colors from the color of the defective pixel, the term “adjacent to” is applied to such physically closest pixels of the same color as the defective pixel.
In the meantime, there are strong demands for improving the electronic image resolution, and the number of imaging element pixels is increasing year by year. However, in the recording or reproduction of moving images or during monitoring of the field with an electronic view-finder (EVF), cases happen, in which particular pixels less in number than the full imaging element pixel number are read out from the imaging element, that is, pixel outputs are read out by thinning out pixels fully or partly. Even in such cases, if defective pixels are contained among the read-out subject pixels, the defective pixel compensation process becomes necessary. In the case of the read-out by thinning out pixels, adequate compensation processing can not be obtained by executing the process with the same algorithm as that applied when reading out the full pixel outputs. This is so because the pixels in horizontal lines vertically adjacent to the horizontal line, to which a defective pixel belongs, have poor correlation property with respect to the defective pixel due to their spaced-apart physical positions from the defective pixel, as will be readily understood in connection with the case of reading out pixel outputs by thinning out pixels in every other horizontal line.
Heretofore, various techniques concerning the pixel defects and the compensation thereof have been proposed. For example, Japanese Patent Laid-Open No. 61-261974, Japanese Patent Laid-Open No. 6-6643, Japanese Patent Laid-Open No. 6-30425, Japanese Patent Laid-Open No. 6-205302 and so forth disclose techniques concerning the detection or determination of defective pixels. As for the defective pixel compensation process, Japanese Patent Laid-Open No. 62-8666 discloses substitution for defective pixel on the basis of normal pixel outputs. Also, Japanese Patent Laid-Open No. 5-236358 discloses compensation circuit and driving thereof with defective pixel compensation pulses. Furthermore, Japanese Patent Laid-Open No. 2000-59799 discloses defect determination method and compensation by substitution. Still further, Japanese Patent Laid-Open No. 2000-228774 discloses compensation using data of lines with relatively less defective pixels. Yet further, Japanese Patent No. 2667938 discloses interpolation of defective pixels in a printer.
Japanese Patent Laid-Open No. 9-247540 proposes techniques, in which, for the case of reading out photoelectric conversion outputs of pixels formed as two-dimensional array on an image pick-up area of an imaging element either by thinning out or from an area restricted with respect to the full image pick-up area, a plurality of different data tables concerning the read-out modes (i.e., data at defective pixel position and positions of normal pixels used for substitution compensation) are preliminarily preserved to permit the defective pixel compensation process to be executed in dependence on a proper table data of which are selectively applied as required, thus reducing the process time.
A digital camera or the like may also have a function of switching two read-out modes, one for fully reading out the pixel outputs of imaging element and the other one for reading out outputs by thinning out the pixels. For example, the former mode is set when obtaining intrinsic recording subject images, and the latter mode is set when continuously monitoring motion images on EVF for selecting the composition or field.
As noted above, in the case of the read-out by thinning out pixels, adequate compensation process can not be obtained by executing the process with the same algorithm as that applied when reading out the full pixel outputs.
However, the above technique proposals all lack the recognition of the technical intent of permitting adequate defective pixel compensation process in correspondence to the switching of the full read-out mode and the thinning-out read mode, and have no mention of means for realizing such intent.